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(54) Title: METHOD FOR DERIVING NOISE-REDUCED ESTIMATES OF COLOUR SIGNAL PARAMETERS FROM 
MULTIPLE COLOUR/LUMINANCE IMAGE SENSOR OUTPUTS 



(57) Abstract 

Color and luminance measurement signals produced by the multiple sensors of a multiband (eg. RGB color) image signal 
processing system are processed to reduce noise. For images such as those obtained from a multi-dye film, the luminance compo- 
nent of the output signal from the luminance sensor may be expressed as a function of the color components of the color sensor 
output signals. A first signal, representative of the constructed low frequency luminance signal, is produced as a linear function 
of the color sensor output signals, employing signal weighting coefficients derivable from intrinsic characteristics of the scanned 
image, such as the respective dye densities of a three layer color film. The luminance measurement signal derived from the lumin- 
ance sensor is subtracted from this first signal, to produce a second signal, which is multiplied by a set f respective scaling fac- 
tors, which are then subtracted from each color sensor output, so as to derive a set of noise-reduced estimates of the color compo- 
nents of each of the color sensor output signals. Each scaling factor is defined in accordance with the noise variances associated 
with the operation of the color image sensors and the luminance image sensor, and is proportional to a product of a correspond- 
ing one of the coefficients and the ratio of the noise variance associated with the operation of a respective color image sensor to a 
summation of the noise variances associated with the operation of each of the color sensors and the luminance sensor. 
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FROM MULTIPLE COLOUR/LUMINANCE IMAGE SENSOR OUTPUTS 

5 FIELD OF THE * INVENTION 

The present invention relates in general to 
color imagery signal processing and is particularly 
directed to a mechanism for- deriving noise-reduced 
color signal estimates from a multiple sensor color 
10 imaging system. 


BACKGROUND OF THE INVENTION 

Multi-channel signal processing systems often 
contain a larger number of sensors than there are bands 

15 of information to be extracted. Namely, they may be 
characterized as containing n sensors for, deriving m 
bands of information, where n>m. For example, a "color 
imagery sensing system, such as employed for high 
definition television, may, on occasion, contain four 

20 image sensors, three of which provide relatively low 
spatial resolution measurements of the red, green and 
blue components of an image, such as the red (R) , green 
(G) and blue (B) density of a color film, while the 
fourth sensor provides a relatively high spatial 

25 ' resolution, luminance-representative (L) output signal. 
A low spatial frequency. luminance signal is generated 
by resampling the original high resolution luminance 
signal, so as to replicate the resolution at which the 
red, green and blue signals are measured. Subtracting 

30 this low frequency luminance from the original 
luminance signal yields only its high frequency 
component. (Aside from this subtraction step,, the low 
frequency luminance information is not used.) The 
resulting high frequency component is then added to the 

35 red, green and blue signals to produce high frequency 
information for these channels. 
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above- referenced three layer film, the luminance 

component of the output signal from the luminance 

sensor may be expressed as a function of the color 

components of the color sensor output signals. Namely, 

5 the luminance sensor output L may be expressed as the 

sum of a function f (r,g,b) and the luminance sensor 
noise n^,, where r, g and b are the color components 

within the color sensor outputs for which reduced-noise 
estimates are desired. 
10 The signal processing mechanism according to 

the present invention processes a vector c± of inputs 
(intrinsic parameters) and a vector c 0 of measurable 
outputs, related by a functxon c 0 = F(c^) which may, in 
some cases, be approximated by a matrix c G = Ac^ . For 

15 a four sensor color, imagery processing system, sensor 

outputs are processed to reduce the noise components ih 
the color sensor output signals and thereby derives 
noise-reduced estimates of each of the color components 
within these sensor output signals. To this end, a 

20 first signal, SI, which is representative of the low 
frequency luminance signal, is produced as a linear 
function of the color sensor output signals, in the 
form SI = (AR + BG + CB) , where A, B and C are signal 
weighting coefficients derivable from intrinsic 

25 characteristics of the scanned image, such as the 

respective dye densities of a three layer color film. 
.The luminance measurement signal derived from the 
luminance sensor is subtracted from this first signal 
SI, to produce a second signal S2. This second signal 

30 S2 is then multiplied by a set of respective scaling 
factors, which are then subtracted from each color 
sensor output, so as to derive a set of noise-reduced 
estimates of the color components of each of the color 
sensor output signals. Each scaling factor is defined 

35 in accordance with the noise variances associated with 
the operation of the color image sensors and the 
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environment that employs a first plurality of n 
measurements on a second plurality of m intrinsic 
variables- Specif ically/ for purposes of the present 
description, the environment is that of a multi-color 
5 image processing system which carries out n-4 

measurements on m=3 intrinsic variables and comprises a 
(multiple sensor) color image section 10, containing 
red/ green, blue and luminance sensitive sensors 13R, 
13G, 13B and 13L arranged on one side of a three layer 
10 color film strip 15, that is illuminated by light 
source 16 having a spectral light intensity I (X) . 
Situated in front of the sensors are respective input 
filters 17R, 17G, 17B and 17L, having respective 
transmittances T r (X) , T g (X) , Tfr(X) and T L (X) . Each 

15 sensor produces image characteristics-representative 
digital signals, i.e. respective red, green, blue and 
luminance channel signal values representative of the 
red, green, blue and luminance components of a color 
image 23 captured on film strip 15 which are coupled 

20 over links 21K, 21G, 21B and 21L to a processor 30. 

As pointed out above, in the course of 
operation of such a multi-channel (e.g. three color 
band, luminance band) image scanning system, the color 
channel sensors 13R, 13G, 13B provide relatively low 

25 spatial resolution measurements of respective red R, 

green G and blue B components of the film, such as its 
red r green and blue densities, while the fourth, 
luminance, sensor 13L provides a relatively high 
spatial resolution luminance representative output 

30 signal. A low spatial frequency luminance signal is 
generated by resampling the original high resolution 
luminance signal, so as to replicate the resolution at 
which the red, green and blue signals are measured. 
Subtracting this low frequency luminance from the 

35 original luminance signal yields only its high 

frequency component. The resulting high frequency 
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processing mechanism of the present invention, it is 
useful to initially define the signal outputs of each 
of sensors 13 of scanning system 10. Specifically, for 
an arbitrary, pixel within each sensor, R, G and B 
5 denote the signals measured by the red, green and blue 
sensors 13R> 13G and 13B, respectively, and L denotes 
the low frequency luminance signal (for the same pixel 
location) for the measured luminance signal. The 
signal values R, G, B and L may be defined as: 


10 

R = r+n r , (1) 

G = g+n g/ . ., . (2) 

B = b+nfc, and (3) 

L = f(r,g,b)+n L (4) 

15 


(Note that this model set of definitions is appropriate 

for scanning a three-layer film, , but not for looking 
at real-world scenes.) Here each noise value n^ is an 

independent observation noise which is normally 
20 distributed with zero mean and variance o^± (tr^ is the 

effective noise variance of the noise in the low- 
frequency luminance signal) . The r, g, and b 
parameters are the "true" or noise-free red, green and 
blue transmittances . The function f is the 

i 

25 (deterministic) relationship between the red, green and 
blue transmit tances and the (low- frequency) luminance 
transmittance. An implicit expression for the function 
f may be obtained by considering the details of the 
scanning process . 

30 The scanner uses a light source 16 (with 

spectral intensity I (X) shining through film 15 
containing three color layers with dye densities dj, d2 
and d3. The optical densities of these dyes, as a 
function of wavelength, are D]. (X) , D2(M arid D3 {X) . 

35 Finally, sensors 13R, 13G, 13B and 13L look at the 
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r = R-(G 2 r /a 2 L ) (f (r,g,b)-L) . df /dr (r, g,b) , . . . (10) 

g = G-(a 2 g /a 2 L ) (f (r,g,b)-L) df /dg (r, g, b) , and . (11) 

b = B-(o 2 b /a 2 L ) (f (r / g\b)-L) . df/db(r,g,b) , . . . (12) 

For a general function f, equations (10), 
(11) and (12) will be non-linear and will not have a 
10 closed form solution. If the function f is linear in 
its arguments, however, its solution can be obtained. 
Thus, if it is assumed that: 


f (r,g,b) = ar + fig + yb (13) 

I 5 

Then, equations (10) , (11) and (12), respectively, 
become: 

r = R-(ar+(5g+yb-L)a(a 2 r /a 2 L ) ........... (14) 

20 

g = G-(ar+pg+Yb-L)p(a 2 g /a 2 L ) . (15) 

b^= B- (ar+Pg+yb-L)y(a 2 b/c 2 L) * - • * < 16) 

25 fbr which the solution is: 

r = R- (aR+pG+yB-b)a(a 2 r /a 2 T ) . • • - . - (17) 

g = G-(aR+PG+yB-L)P(a 2 g/a 2 T ) (18) 

b = B- (aR+pG^yB-L)y(o 2 b/o 2 T ) (19) 

30 

where: 

o 2 T = a 2 L + a 2 o 2 r + p 2 o 2 g + y 2 o 2 b . (20) 
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then the rms error of the estimate becomes (after 
algebraic simplification) : 

° 2 M = 0 2 M-(aqa2 r+ p SCT 2 g+y ta2 b) 2/ 0 2 T (25) 

5 ' 

which is always the better estimate. 

In accordance with a preferred embodiment, 
the system will not be light-limited, so that the noise 
level from each sensor will be the same. The low- 
10 frequency luminance signal is obtained by averaging 

four luminance measurements, so that its rms noise is 
half that of the other channels «7 r = o g = o b = 2ct l ) . 
For exemplary parameters a = .3, 0 = .6, and y = .1, 
equations (17), (18) and (19) yield: 

15 

A 

r = R-.42A with ct£ = (.93)a r , ....... .... (2 6) 

g = G-.85A with aA = (.70>a a , and 


20 b = B-.14A with CT£ = (.99)O b , 


g = i./ujog, ana ; (27) 

- • - - . . ' . (28) 


where: 

• "> 

A = -3R + .66 + .IB » L- . . . 
25 < * ( y| 

By slightly modifying the coefficients to: a = .4, 
0 = .5, and y = .1 the values of r, g and £ become/ 


r = R-.60A with G£ = (.87)a r , . . . . . , ... . , (30) 

30 

g = G-.75A with a£ = (.79)o g , and (31) 

b = B-.15A with a£ = (.99)o b , (32) 
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measurement noise. If the film noise (k r ,k g ,k b ) is 
additive in intensity space, then the measurements may 
be defined as: 


5 R = r + k r ^ n r , (34) 

G = g + kg + n g/ (35) 

B = b + k£> + n b , and .. (3 6) 

L = <x(r+k r ) + P(g+k g ) + y(b+k b ) + n L (37) 

10 If the film noise is additive iri density space, the 
definitions are: 

R = rlO^ + n^, (38) 

G = gl0*s + n g , (39) 

15 B = blO^h + n b , and r , (40) 


L = a(rl0 )tr ) + P(glO k 9) yCblO**) + n L - . . .. . (41) 

In both cases the maximum likelihood estimators are 
unchanged from the simpler case discussed previously. 
20 Conseguently / for the additive in intensity space case, 
the expression for r becomes: 

r> = R-(OR+PG+YB-L)a(Q 2 r /o 2 T ) (42) 

= - r+k r +n r - (an r +png+7n] C) -n Ij ) a (o 2 r /o 2 T ) (43 ) 

25 

Similar expressions may be derived for g and b. It 
should be observed that contributions of kg and k^ to r 

come from L as well as from G and B, but that these 
contributions cancel. Thus, the only increase in the 

A 

30 rms error of r is the expected contribution from the 

film noise in the red layer. The same result holds for 

the case of noise which is additive in density space, 

and it also holds for linear combinations of red, green 

and blue signals.. That is, 

35 
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green, and blue signals. The actual magnitude of the 
reduction depends on the relative noise levels in the 
different; sensors and on the details of the 
relationship between the luminance signal and the three 
color signal's. (m the exemplary, but realistic case, 
a reduction of green measurement noise by 20-30% can be 
obtained.) if the graininess of the film is included 
in the noise considerations, the proportional reduction 
an noise will be smaller, but there are no 
contributions to noise in one channel due to film noise 
in another. 

While I have shown and described an 
embodiment in accordance with "the present invention, it 
is to be understood that the same is not limited 
thereto but is susceptible to numerous changes and 
modifications as known to a person skilled in the art, 
and I therefore do not wish to be limited to the 
details shown and described herein but intend to cover 
all such changes and modifications as are obvious to 
one of ordinary skill in the Art. ' , 
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2. A method according to claim 1, wherein 
step (b) comprises generating said second signal as the 
difference between said first signal and said further 
sensor output signal. 

5 

3. _ A method according to claim 2, wherein 
step (c) comprises subtracting a fraction of said 
second signal from each of said first sensor output 
signals to thereby obtain said noise-reduced estimates. 

10 of the information components of each of said first 
sensor output signals. 

4 . A method according to claim 1 , wherein 
step (c) includes generating a plurality of third 

15 signals representative of the noise variances 

associated with the operation of said first image - 
sensors and said further image sensor/ and combining 
the first sensor output signals produced by said first 
image sensors with the second signal generated in step 

20 (b) and said plurality of third signals, to thereby 

obtain said noise-reduced estimates of the information 
components of each of said first sensor output signals. 
~j> * 

5. A method according ta claim 4, wherein a 
25 respective one of said third signals is representative 

of the ratio of the noise variance associated with the 
operation of ja respective first image sensor to a 
summation of the noise variances associated with the 
operation of said plurality of first image sensors and 
30 said further image sensor. 

6. „ A method according to claim 1, wherein 
said plurality of first image sensors produce first 
sensor output signals associated with respectively 

35 different color components of an image, and wherein 
step (a) comprises generating said first signal as a 
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operation of said plurality of first image sensors and 
said further image sensor, and combining the first 
sensor output signals R, G and B produced by said first 
image sensors with the second signal S2 generated in 
5 step (b) and said plurality of third signals, to 
thereby obtain said noise-reduced estimates of the 
information components of each of said first sensor 
output signals. 

10 11. A method according to claim 10, wherein 

a respective third signal, associated with one of said 
color components, is proportional to a product of a 
corresponding one of said coefficients and the ratio of 
the noise variance associated with the operation of a 

15 respective first image sensor to a summation of the 

noise variances associated with the operation of said 
plurality of first image sensors and said further image 
sensor* 

20 12. For use with a multiple band image 

sensor system having a plurality of first image 
sensors, each of which produces a respective first 
sensor output signal that contains an information 
component representative of a respective characteristic 

25 of said image and a noise component associated with the 
operation of the sensor through which its output signal 
is produced, ..and a further image sensor that produces a 
further sensor output signal containing an information 
component that is representative of a further 

30 characteristic of said image, and is expressible as a 
function of the information components of said first 
sensor output signals, and a noise component associated 
with the operation of said further image sensor, a 
method of processing said first and further sensor 

35 output signals to reduce the noise components in said 
first sensor output signals and thereby derive noise- 
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15. A method according to claim 14, wherein 
said further image sensor produces said further sensor 
output signal in accordance with a luminance component 
of said image, and wherein step (c) comprises 

5 generating a" third signal as the difference between 
said first signal and said luminance component 
representative further sensor output signal and 
subtracting from said first sensor output signals 
respective combinations of said second and third 
10 signals. 

16. A method according to claim 15, wherein 
said plurality of first image sensors produce first 
sensor output signals R, G and B, respectively 

15 associated with red, green and blue color components of 
an image, and wherein step (a) comprises generating a 
first signal SI as a linear function of the respective 
red, green and blue color representative first sensor 
output signals, in accordance with the relationship: 

20 SI = aR + bG + cB, 

where a, b, and c are prescribed coefficients.. 

> 17. A method according to claim 16, wherein 

said- further image sensor produces a further sensor 
25 output signal L in accordance with a luminance 

component of said image, and wherein step (c) comprises 
generating a third signal as the difference between 
said first signal SI and said luminance component 
representative further sensor output signal L. 

30 

18. A method according to claim 17, wherein 
step (c) includes generating a plurality of second 
signals, a respective one of which is representative of 
the ratio of the noise variance associated with the 
35 operation of a respective first image sensor to a 

summation of the noise variances associated with the 
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(a) generating a first signal, 
representative of said further image sensor signal, as 
a function of said first sensor output signals; 

(fc>) generating a second signal 
5 representative of a prescribed relationship between 
said first signal and said further sensor output 
signal; and 

(c) combining the first sensor output 
signals produced by said first image sensors with said 
10 second signal generated in step (b) , to obtain said 

noise-reduced estimates of the color components of each 
of said first sensor output signals, 

21. A method according to claim 20, wherein- 
15 step (b) comprises generating said second signal as the 

difference between said first signal and said further 
sensor output signal, and wherein step (c) comprises 
subtracting a fraction of said second signal from each 
of said first sensor output signals to thereby obtain 
20 said noise-reduced estimates of the color components of 
each of said first sensor output signals, 

22. A method according to claim 20 r wherein 
step (c) includes generating a plurality of third 

25 signals representative of the noise variances 

associated with the operation of said first image 
sensors and said further image sensor, and combining 
the first sensor output signals produced by said first 
image sensors with the second signal generated in step 

30 (b) and said plurality of third signals, to thereby 
obtain said noise-reduced estimates of the color 
components of each of said first sensor output signals, 

23. A method according to claim 22, wherein 
35 a respective one of said third signals is 

representative of .the ratio of the noise variance 
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sensor output signals R, G and B produced by said first 
image sensors with the second signal S2 generated in 
step (b) ^nd said plurality of third signals, to 
thereby obtain said noise- reduced estimates of the 
5 color components of each of said first sensor output 
signals . 

27. A method according to claim 26, wherein 
a respective third signal, associated with one of said 

10 color components, is proportional to a product of a 

corresponding one of said coefficients and the ratio of 
the noise variance associated, with the operation of a 
respective first image sensor to a summation of the 
noise variances associated with the operation of said 

15 plurality of first image sensors and said further image 
sensor. 
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